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Abstract: Student's emotional aspects are often discussed in order to promote better learning activity in blended learning 
courses. To observe these factors, course participant's self efficacy and reflections upon their studies were surveyed, in 
addition to the surveying of the metrics of student's characteristics during a Bachelor level credit course. Regarding the 
causal relationships between these factors, the contributions of the factors of self efficacy and other characteristics were 
evaluated. The contents of notes students took during the course were lexically evaluated to determine whether this 
activity promoted reflection. Four indices of note-taking activities were extracted from the lexical analysis. Correlation 
analysis was conducted, and according to the provisional results of the correlation analysis between the four indices of 
note-taking and student's characteristics of their own degree of self efficacy, there were some significant relationships 
between note-taking indices and some of the self assessment indices, such as word rates in notes and the degree of out of 
course study, and between the content coverage of notes taken and self understanding. 
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1 Introduction 

Various types of e-learning are proliferating, and becoming more widely used due to their beneficial 
educational aspects (Hill, 2012). In particular, blended learning, which consists of face-to-face sessions and 
learning materials that are supported by information communication technologies (ICT), is the easiest way to 
use modern educational media which is familiar to both students and lecturers (DesLauriers et al. 2011). 
Conventionally, when the relationship between student's learning activities and achievement has been 
discussed, learning performance in an e-learning environment was always a more important topic than 
participant's satisfaction. Since encouraging student's learning activities has been shown to improve 
performance, the e-learning environment, which uses learning materials together with ICT, is considered to 
contribute positively to their results. Recently, flipped classrooms, which are a kind of a blended learning, are 
believed to promote student's self directed learning outside of classroom and to help collaborative learning in 
face-to-face sessions (DesLauriers et al. 2011). 

To observe student's learning activity during courses, two types of learning evaluation have conventionally 
been used: emotional, and cognitive evaluation (Bloom et al. 1971). In addition to the use of learning 
motivation studies (Miyamoto, Nasu, 1995) to evaluate emotional aspects, evaluation of student's preferences 
and their level of satisfaction, such as through the use of the ARCS model (Keller, Suzuki, 1988), is discussed. 
Regarding students' attitudes towards learning, their learning efficacy and information which is based on self 
reflection are often considered (Pintrich 1990). For cognitive assessment, note taking activity has been 
observed in order to track student's learning progress (Kiewra, 1989; Kobayashi, 2005; Nye et al. 1984; Tynaja, 
1999). In addition to recording the content of notes taken, a lexical illustration of notes enables the use of 
conceptual mapping representation (Novak and Canas, 2008). As a result, the effectiveness of learning 
performance can be confirmed, even during online courses (Nakayama et al. 2014a; Nakayama et al. 2014b). 

In the past few years, more detailed analysis of participant's learning activity has been promoted, in order to 
improve and enhance learning performance. This is often referred to as learning analytics, and it can provide 
significant feedback to stakeholders (Vahdat et al. 2015). Some of the techniques have been developed using 
information based on conventional sources, such as student's characteristics and learning activities (Greller 
and Drachsler, 2012). In addition to improving learning performance, learning analytics provides a design 
process for learning, using an instructional design technique to optimise learning activity (Greller and 
Drachsler, 2012). Regarding the above discussion, a record of note-taking activity can be a source of learning 
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analysis. Also, the emotional effectiveness of note taking activity in a blended learning environment can be 
observed. 

The purpose of this paper is to examine the relationship between participant's evaluations of their own self- 
efficacy and their note-taking activities. Towards this aim, two sets of questionnaires were used to survey 
student's self reflection, and their note taking activities were lexically analysed. 

The following topics are addressed in this paper: 

■ Student's self efficacy and the ways in which they reflect upon their own learning activities were 
measured and evaluated. 

■ The relationship between student's self efficacy and student's characteristics were causally analysed. 

■ Note-taking activities were evaluated using metrics based on lexical analysis of both the lecturer's 
presentations and the contents of student's notes. 

■ The relationships between measurements of participant's self assessments and metrics of note-taking 
activity were examined in order to confirm the effectiveness of note-taking behaviour. 

2 Method 

2.1 Blended learning course as a survey course 

The surveys were conducted during a blended learning course at a Japanese university. The course consisted 
of 15 weeks of face-to-face sessions. The course was a Bachelor level Information System Network credit 
course (Nakayama et al. 2011). For this cohort, several conventional surveys were conducted, to evaluate 
participant's characteristics at the beginning of the course. Some metrics were also surveyed in the middle of 
the course. In addition to these, two types of surveys of student's reflections were employed for this study. 

To monitor participant's learning progress, all participants were asked to present the notes they took during 
each session of the course. These notes were scanned, and the images were stored on a PC. The textual 
content of the notes was lexically analysed. 

To encourage participants to take notes, some note-taking techniques were provided, using examples of well- 
taken notes. Instructions were given twice during the course, once early in the course and again at the 
midpoint. 

The valid number of participants for surveys was 40, but the number of valid participants for note content 
analysis was 27. 

2.2 Characteristics of students 

In this study, student's characteristics, such as Personality (Goldberg 1999; I PI P 2004), Information Literacy 
(Fujii 2007), Note taking skills (Nakayama et al. 2014a) and Learning Experience (Nakayama et al. 2007) were 
continuously surveyed (Nakayama et al. 2007; Nakayama et al. 2014b, 2016). These metrics are introduced 
here. 

Personality: Five factor scores were extracted using a public domain item pool, the International Personality 
Item Pool (IPIP) inventory (IPIP 2004). The five components are “Extroversion" (IPIP-1), “Agreeableness" (IPIP- 
2), “Conscientiousness" (I PI P-3), “Neuroticism" (I PI P-4) and "Openness to Experience" (I PI P-5). 

Information Literacy: Information literacy inventories (32 items) were defined and developed by Fujii (2007). 
Originally, 8 factors were extracted, and they can be summarised as two secondary factors: Operational Skills 
(IL-1), and Attitudes towards Information Literacy (IL-2) (Nakayama et al. 2008). 

Learning experience: Students' online learning experiences were measured using a set of questions, and three 
factors were identified, as follows. Factor 1 (LE-F1): Overall Evaluation of the e-learning experience, Factor 2 
(LE-F2): Learning Habits, and Factor 3 (LE-F3): Learning Strategies (Nakayama et al. 2007). 
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Note-taking skills: Student's note taking skills were measured using the following three factors (Nakayama et 
al. 2014a). They are NT-F1: Recognition of functions of note taking, NT-F2: Methodology of utilising notes, and 
NT-F3: Presentation of notes. 

2.3 Participant's reflections upon learning activity 

Participant's emotional factors and impressions of their leaning attitudes during the course were surveyed, 
using two sets of questionnaires. The first one is a self efficacy metric consisting 9 question items which were 
developed by Pintrich (1990), as shown in Table 1. The second one is a metric of participants' level of 
satisfaction regarding their experience during the course, such as the participant's self directed effort. 
Student's self assessment of the degree of effort, self satisfaction, and study hours are frequently included in 
course assessments. These types of questions are often used in participant's assessments of courses. Some of 
them were measured using a 5 point scale, and others were measured using a 10 point scale. 

Table 1: Questionnaire items regarding self efficacy 



Question item 

Mean 

Factorl 

Factor2 

2 

I'm certain 1 can understand the contents taught in this course 

3.75 

0.86 

-0.03 

1 

1 expect to do very well in this class 

3.73 

0.79 

-0.17 

5 

Compared wilh others in this class, 1 think I'm a good student 

3.28 

0.74 

0.14 

4 

1 think I'll receive a good grade in this class 3.08 

3.08 

0.70 

0.21 

6 

1 am sure 1 can do an excellent job on the problems and tasks assigned for this class 

3.60 

0.51 

0.20 

8 

Compared with other students in this class 1 think 1 know a great deal about the subject 

2.65 

-0.27 

0.82 

7 

My study skills are excellent compared wilh others in this class 

2.85 

0.18 

0.59 

3 

Compared with other students in this class 1 expect to do well 

3.10 

0.25 

0.48 

9 

1 know that 1 will be able to learn the material for this class 

3.38 

0.13 

0.42 

Contribution ratio (of each factor with other factors eliminated 


0.31 

0.17 

Contribution ratio (of each factor with other factors ignored) 


0.33 

0.19 


2.4 Lexical comparison of lecturer's presentations and student's notes 

The contents of participant's notes were read and recorded manually, as computer readable text. The 
lecturer's hand-written notes to be presented to participants during face-to-face sessions were also 
transformed into computer readable text. Notes of both the participants and the lecturer were lexically 
analysed using the Japanese morphological term analysis tool MeCab. Nouns were extracted from the texts of 
the notes. From these, term-session matrices, such as frequency of nouns across sessions, were generated. 
The term frequencies in the contents of notes of both the lecturer and the participants were evaluated as 
follows (Nakayama et al., 2014a, 2016). 

■ Word ratio: the ratio between the number of terms written and the number of terms given (the number 
of terms participants recorded vs. the number of terms the lecturer presented). 

■ Coverage: the coverage ratio was calculated as a percentage of the number of terms recorded by 
participants. 

To extract the semantic structures in the contents of note taken, a social network analytical technique was 
used on the texts of notes (Jin, 2009; Rabbany et al. 2011). In comparing the contents of the lecturer's 
presentations with the student's notes, co-occurring nouns were analysed using a previously reported 
methodology (Nakayama et al., 2014a, 2016). Term co-occurrence shows the structure of the conceptual 
meanings using a lexical representation of the term connection patterns. Noun transitions in the notes were 
extracted from phrases, such as A-B and B-C extracted from the text A-B-C. The relationship between the two 
terms, known as 2-gram nouns, is summarised using an adjacency matrix. An example of the lecturer's 
presentation in Session 13 is shown in Figure 1. The matrix of the participants should coincide precisely with 
the lecturer's matrix when all contents have been transferred to the participants. The adjacency matrix can be 
illustrated as a networked graph, such as a conceptual map (Novak, 2008; Rabbany et al. 2011). The difference 
between the two maps shows the distinctness in processing information between the lecturer and the 
participants. Therefore, the differences indicate the degree of transformation of the lecturer's contents. The 
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differences between the two matrices can be calculated as a distance measure. The distance between the 
lecturer's presentations and student's notes is defined using two metrics, as follows: 

■ Additional Distance means the sum of the number of additional nodes or edges in a matrix. 

■ Insufficient Distance means the sum of the number of reduced nodes or edges in a participant's matrix, in 
comparison with the lecturer's matrix. 

Both distances are influenced by the total number of terms in the lecturer's presentation, so that the relative 
distances are calculated using the number of terms the lecturer presented in each session. 


As a result, note-taking activity was evaluated using four indices of note-taking, then overall averages across all 
sessions and partial averages for the first and the second halves of sessions were calculated, respectively. 
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Figure 1: Example of adjacency matrix (Left) and the relationship between two adjacency matrices (Right) 


2.5 Causal relationships analysis across the indices 

The relationships among the indices mentioned in the above sections were examined using a structural 
equation modelling technique (SEM). The possible causal relationships and the parameters of the models were 
estimated using structural equation modelling software (AMOS) (Toyoda 2007), and the validity of the models 
was tested using indices of the fitness of the model (the GFI: Goodness of Fitting index). 

3 Results 


3.1 Responses of participant's self reflection 

The means of participant's responses for the two sets of questionnaires are summarised in two tables. Table 1 
represents self efficacy and Table 2 represents their own reflections during the course. 


All means for questions concerning self efficacy inventories are above the middle value of a 5 point scale, 
indicating that participants have responded positively. As these means are at the same levels, the latent 
factors are extracted using factor analysis with Promax rotation. The factor loading values for the two factor 
structures are also summarised in Table 1. As the table shows, the first factor contributes over 30% of the 
total. Therefore, regarding the contents of the question items, the label for the first factor (SE-1) is "self 
confidence in student's own attitude", and the label for the second factor (SE-2) is "self confidence in student's 
own level of competence". The mean scores for the two factors are displayed in Table 2, where the score of 
the first factor (SE-1) is higher than the score of the second factor (SE-2), while their correlation coefficient is 
small (r=0.13). The participants have confidence in their own attitudes rather than in their levels of 
competence. 


The levels for self evaluative responses are also relatively high, as shown in Table 2. These results suggest that 
most students participate sincerely, and they satisfy the requirements of the course. The exception is 
participant's "learning hours", a question regarding their own learning opportunities outside of the classroom. 
This point is often noted by researchers of higher education (Kaneko, 2009). 


The responses mentioned in the two tables correlate with each other, and these responses may also be related 
to the student's characteristics which were mentioned in the above sections. 
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Table 2: Question items and means for self evaluation 



Label 

Question Items 

Mean 


SE-1 

Factorl: self confidence in student's own attiutude 

3.49 


SE-2 

Factor2: self confidence in student's own level of comptence 

2.87 

1 

Syllabus reading 

Read the syllabus of this course carefully in advance 

3.65 

2 

Out-of-class study 

Study out of class for this course 

2.83 

7 

Learning hours 

Grade a level of learning hours for this course oul of the class 

2.38 

3 

Self directed effort 

Self assessment of own altitude towards this course 

7.26 

4 

Self understanding 

Self assessment of level of understanding for contents of this course 

6.60 

5 

Self achievement 

Self assessment of level of achievement for this course 

6.53 

6 

Self satisfaction 

Self assessment of level of overall satisfaction lor this course 

7.60 


Bold means are based on a 10 point scale 


3.2 Causal relationships across the indices 

3.2.1 Learning activity outside the classroom 

In regards to the above discussion, learning hours and frequency of study outside the classroom should be 
measured, in order to evaluate student's self directed learning activity. A possible causal relationship is 
displayed in Figure 2. Significant path coefficients are indicated using bold characters, and non-significant 
values are represented by (). 



Figure 2: Relationships between some student's characteristics and learning opportunity outside the classroom 

Some student's characteristics, such as information literacy (IL-1 and IL-2), learning strategy (LE-F3), and some 
self evaluation inventories were selected. Since the subject of this course concerns information systems, the 
information literacy of operational skills (IL-1) affects the extent of syllabus reading, while student's learning 
strategy experience affects the degree of self directed effort. Both contribute to student's level of attention 
towards learning outside the classroom, which is shown in Figure 2 as causal paths. This confirms the 
importance of both lecture syllabi, and the degree of effort students make, as reflected in their number of 
hours of study. 

3.2.2 Relationship between self efficacy and reflections 

In the previous section, two types of metrics were introduced to evaluate participant's emotional factors, such 
as self efficacy and reflections. The means are summarised in Table 2. 

As mentioned in Figure 2, information literacy affects participant's attitude, thus it has also been employed in 
this analysis. The contribution of information literacy and the relationships between self efficacy and 
reflections are summarised in Figure 3, using a path diagram. Though some path coefficients are not 
significant, the goodness of fit index (GFI) of the structure of this model is significant (GFI=0.87). 
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Figure 3: Contribution of information literacy to self efficacy and self assessment. 

The factor of information literacy (IL-1) affects the factor of self efficacy, such as confidence in one's own 
attitude (SE-F1) and self directed effort. Both self efficacy (SE-F1) and self directed effort significantly affect 
self satisfaction, while self directed effort also affects student's own impression of their level of self 
understanding and indirectly affects self achievement. This causal path suggests that participant's confidence 
in their attitude and self directed effort have an effect on their self evaluation, such as the level of satisfaction, 
and the degree of understanding of the course contents. 

3.2.3 Contribution of note-taking skills and learning experience 

As mentioned in the Introduction section, note taking activity may have some effect on participant's emotional 
factors. As factors of student's learning experience have contributed to learning activity outside of the 
classroom, some additional factors may also affect their self evaluations. 

To validate the hypothesis, factors for note taking skills and factors for learning experience were mapped in 
Figure 3 using causal paths. The contribution of note taking skills was confirmed, as Figure 4 shows. Though 
possible paths which concern note taking skills have been indicated, most are not significant. 



Figure 4: Causal relationships while note-taking skills were being introduced 

Additionally, factors for learning experience were introduced into the causal path diagram, as shown in Figure 
5. Though some of the factors, such as LE-F1 (e-Learning experience) and LE-F3 (Learning strategies) 
significantly influence participant's self evaluation, the GFI of this model is not significant (GFI=0.79, AGFI=0.64, 
RMSEA=0.10). 
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Figure 5: Causal relationships between self efficacy and reflection using factors of student's characteristics 

Therefore, the contribution of participant's learning activities, such as note taking skills and student's learning 
experience towards their own self-assessment was not confirmed. To verify the hypothesis, a more detailed 
analysis was employed, such as making changes to participant's activities as the course progressed, and in the 
metrics of the contents of notes taken. 

3.3 Note taking activity 

Four types of indices for note taking activity were measured, using both the lecturer's presentations and 
student's notes from each classroom session across the 14 weeks of the course. As the number of terms 
depends on the contents of the lecturer's presentation, word rates and distances are normalised using the 
lecturer's metrics. In order to consider student's experience, the metrics between the first and second halves 
of the course were compared, such as the means of the first 7 sessions and the means of the second 7 
sessions. 



Figure 6: Mean metrics of note-taking activity across the course 1st and 2nd halves 

Mean ratios are summarised in Figure 6, which compares these two ratios. Though word rates in the second 
half decreased in comparison with sessions in the first half, both means of lengths for sessions in the second 
half are higher than for means in the first half. There are significant differences in word ratios, and there are 
insufficient distances and additional distances (p<0.05) between the 1st and 2nd halves of the course as well. 
These results suggest that during the course, as students gradually take notes which use their own 
terminologies instead of those presented by the lecturer, the overall number of words decreases. Note-taking 
activity is based on student's note-taking skills. The factor scores for note-taking skills between the two halves 
of the course were compared, and the ratios summarised in Figure 7. As all three factor scores increased 
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during the sessions in the second half, these skills may have been sharpened by the lecturer's instructions 



Figure 7: Mean factor scores of note-taking skills in the two surveys 

Table 3: Correlation coefficients between differences in note-taking metrics and factor scores 



d-WR 

d-CV 

d-ID 

d-AD 

d-NT_F1 

0.04 

-0.09 

0.18 

-0.12 

d-NT_F2 

0.56 

0.22 

0.03 

0.42 

d-NT_F3 

0.39 

0.09 

0.12 

0.32 


Correlation coefficients were calculated between the differences in note metrics and the differences in factor 
scores of note-taking skills. These coefficients are summarised in Table 3. Both differences between F2: 
Methodology of utilising notes and F3: Presentation of notes correlate with the differences in word rates. The 
enhancement of these note-taking skills improves the number of words written down. Therefore, note-taking 
skill instructions affect metrics of note-taking activities. 

3.4 Relationship between note-taking activities and student's self reflection 

As mentioned in the Introduction section, it is hypothesised that note-taking activity may affect student's 
emotional aspects, such as self reflection. Though the effectiveness of student's characteristics on improving 
self evaluation was tested in the above section, the significance of the contribution was not determined using 
causal relationship analyses. 

Therefore, metrics for note taking activities are introduced instead of student's characteristics, since these 
reflect student's learning activity during the course. 

The causal relationship illustrated in Figure 8 indicates paths to student's self evaluations using the distances 
between student's notes and the lecturer's presentations. The GFI is sufficient to permit acceptance of this 
model. Both distances affect the two metrics of notes, namely word ratio and content coverage, in a 
conventional manner. Also, the additional distance impacts the degree of self satisfaction. Most metrics for 
note-taking affect self evaluations positively. However, the coverage for note-taking negatively influences the 
evaluation, as it means restricting student's own written self evaluations. Regarding this relationship, note¬ 
taking activity contributes to student's self evaluations regarding understanding and satisfaction in the course, 
for example. The causal relationships between the 1st and 2nd halves were compared using the metrics. 
Though a modified model can be produced for sessions in the 1st half, the model is not significant for the 
sessions in the 2nd half. The student's process of improvement should be tracked in order to consider changes 
which might improve student's self evaluations. Therefore, the gathering of additional data during further 
surveys will be required in order to determine the validity of students' self evaluations in the second half of the 
course. 
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GFI=0.94, AGFI=0.86, RMSEA=0.00 

Figure 8: Causal relationships between metrics of note-taking activity and student's self evaluation 

4 Discussion 

Student's emotions, such as motivation and self efficacy are a key factor in the promotion of learning and 
understanding (Miyamoto and Nasu, 1995). In particular, the factor of self efficacy contributes their learning 
activity (Pintrich and Goot, 1990). To understand the details of the concept of self efficacy, participant's 
responses were analysed using factor analysis, and two factors were extracted from a survey of student's self 
efficacy, namely self confidence in one's own attitude and one's own level of competence at the beginning of a 
course. The mean scores of the two self efficacy factors and the reflection indices represent student's 
evaluation of their learning in the survey course. Since most scores were over the median of the scale, the 
course format was encouraging to most participants, and they enjoyed taking the course. 

The causal relationships between these emotional factors and student's characteristics, which were surveyed 
in order to understand their learning activity and their behaviours, were examined. The insufficiency of 
student's outside-of-the-classroom learning activity in comparison with the lecturer's expectations, in the 
context of conventional learning environments, has been widely discussed. Unfortunately, the existence of the 
same phenomenon in this blended learning course was confirmed, even though the course has employed 
various procedures, such as taking online tests, to encourage participants to do more outside of the classroom. 
In a causal analysis, the number of hours students spent studying, which is an indicator of self directed 
learning, is explained by the extent of their syllabus reading and by some of the student's characteristics. This 
result may suggest that, during course design, assistance for students which considers their characteristics is 
required. Also, factors of information literacy contribute to both aspects of self efficacy and self evaluation. 

In order to determine student's learning activity during the course, the contents of their notes were analysed, 
and those contributions were examined. During the course, student's recognition of the need for better note 
taking skills improved, resulting in increases in the factor scores in the two surveys. The instructions regarding 
note-taking techniques may encourage most participants to take better notes. The results of correlation 
analysis of factor score differences between note-taking skills and indices of note-taking activity reveal that 
participants understanding of note-taking skills improved the contents of their notes. The results of causal 
analysis, which consisted of four indices of notes taken and student's self reflections, show some significant 
causal relationships. For example, "additional distance" positively affects "self satisfaction", while "word rate" 
positively affects "learning hours". Regarding these relationships, encouragement of note-taking during a 
course may be a positive influence on participant's self reflection. However, a more detailed analysis is 
required before any generalisations can be made. This analysis will be a subject of our further study. 

5 Conclusion 

The relationships between participant's own reflections and note-taking activities during a blended learning 
course were analysed, to examine the effectiveness of both learning performance improvements and note 
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taking activity upon emotional factors related to learning. Towards this aim, various metrics of student's 
characteristics and the contents of notes they took were examined. 

Regarding the purposes of this paper, the results are summarised as follows. 

• Participant's self assessments and reflections during a blended learning course were measured and 
evaluated. Most responses were positive, with the exception of the results for learning hours outside 
of class. 

• The relationship between student's self efficacy and student's characteristics was causally analysed. 
Regarding the relationship, it was confirmed that the level of information literacy and student's 
learning experience contributed to factors of student's reflection. 

• Note-taking activities were evaluated using four metrics, such as word ratio, coverage, additional 
distance and insufficient distance. The activities were quantitatively assessed during the course. The 
relationships between these evaluations and student's own evaluations were analysed. 

• Using causal analysis, the relationships between the measurements of participant's self assessments 
and the metrics of note-taking activity were examined. In this analysis, the note-taking activities were 
shown to have been influenced by student's reflections. 

The results indicate that note-taking activities significantly affect the level of self-assessment that is based on 
these reflections. In addition to this modelling, the study of learning behaviour is also discussed in order to 
improve learning performance. 
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